Video quality analyzer

ABSTRACT

Methods, systems and software are disclosed for automated Measurement of Video Quality parameters. The system includes a static Test Pattern provided either in form of a Test Pattern File, converted via a standard playout device (test source) into analog or digital test signal and supplied to the input of a System Under Test, or in form of a Reflectance Chart installed before the front-end device of the System Under Test, such as TV camera. The system also includes a video capture device connected to the back-end device of the System Under Test, e.g. to the output of system decoder/player. A Video Quality Analyzer processes the captured video data and generates a detailed Analysis Report.

This application is a continuation of Ser. No. 13/225,476, filed Sep. 4,2011, the content of which is incorporated by reference.

BACKGROUND

With the introduction of advanced digital delivery systems for audio andvideo, there is an increased awareness of the relationship betweensubjective (perceived) quality and objective (measured) quality of videoimages presented to the observer's eye. Video quality is acharacteristic of a video passed through a video transmission/processingsystem, a formal or informal measure of perceived video degradation(typically, compared to the original video). Video processing systemsmay introduce some noticeable amounts of distortion or artifacts in thevideo signal, so video quality evaluation is an important problem.

Currently there are many tools for analyzing video quality utilizing theFull Reference Methods (FR) such as dual-stimulus methodology based oncalculation of differences between original and processed video data andsubsequent transformation of these differences in accordance withpredetermined metrics.

Typically, objective methods are often classified based on theavailability of the original video signal, which is considered to be ofhigh quality (generally not compressed). These metrics are usually usedwhen the video coding method is known. PSNR (Peak Signal-to-Noise Ratio)is the most widely used objective video quality metric. However, PSNRvalues do not perfectly correlate with a perceived visual quality due tonon-linear behavior of human visual system. The PSNR calculation on thepre-selected set of live clips is very long and tedious job, so in factit is executed only during acceptance test of some large-scale systems.In other words, this methodology is not suitable for fast measurement oflarge quantity of different video processors and/or processingmodes/profiles. More sophisticated metrics require even morecalculations, thus they are even less suitable for fast objectivemeasurements.

Moreover, PSNR compression artifacts metering implies that both A and Bpicture have same resolution, horizontal and vertical positions, videolevels and (very important)—same frequency response, i.e. both picturesare perfectly aligned in space and time. Only under these conditionsPSNR reading correlates well with subjective quality estimates. Inmodern content delivery systems such conditions are very seldomsatisfied.

A second approach is represented by well established techniques ofmeasuring objective video processing parameters on some artificialmatrix test pattern. This approach captures video data and subsequentlyanalyzes the captured video data. However, automatic video analyzers inthis approach suffer from lack of flexibility: they are limited to ashort list of video image resolutions and signal formats—any imagesize/position/resolution deviation from perfect match results in afailure of the analysis process. Additionally, analysis of pre-captureddata files is not supported. With application to the analysis of videocameras performance, analyzers of this kind provide mainly waveformmonitor functionality, i.e. only manual controls, thus excluding anyautomated analysis.

A third approach is represented, for example, by IE-Analyzer made byImage Engineering, Gmbh in Germany. This automated hardware/softwaretool is suitable for accurate and detailed camera performance analysis,but requires a nearly perfect setup of lighting conditions and camera'span/zoom/tilt controls. IE-Analyzer can work with pre-captured files,but positioning of dotted lines delimiting the ROI (Region Of Interest)should be done manually. Moreover, for each reported parameter adifferent reflectance test chart or test pattern transparency isrequired, so the complete measurement process takes a long time, andnearly perfect studio conditions and highly skilled technical personnelare pre-requisites.

SUMMARY

In a first aspect, methods, systems and software are disclosed forautomated Measurement of Video Quality parameters. The system includes astatic Test Pattern provided either in form of a Test Pattern File,converted via a standard playout device (test source) into analog ordigital test signal and supplied to the input of a System Under Test, orin form of a Reflectance Chart installed before the front-end device ofthe System Under Test, such as TV camera. The system also includes avideo capture device connected to the back-end device of the SystemUnder Test, e.g. to the output of system decoder/player. A Video QualityAnalyzer processes the captured video data and generates a detailedAnalysis Report.

In a second aspect, a video monitoring system to perform automatedMeasurement of Video Quality parameters includes a static test patternprovided as a test pattern file or a reflectance chart, the test patternfile rendered by a device under test, the reflectance chart capture by acamera under test, wherein the device under test or the camera undertest generates video data for analysis; and a video quality analyzerprocessor processes the video data into detailed analysis report.

Implementations of the above aspects may include one or more of thefollowing. The Test Pattern contains video components equally suitablefor (1) aural and visual estimation, (2) for on-line or off-lineinstrumental analysis, and (3) for fully automated on-line or off-lineanalysis. The Test Pattern components include several horizontal TestBands, forming multi-row matrix, each band containing test patterncomponents specific for the particular sub-set of video qualityparameters, such as video levels, frequency response, pulse response,etc., thus providing for a multitude of video test components combinedin one test pattern, e.g. the said Test Pattern includes Test Bandsconsisting of (1) Color Bars, (2) Inverted Grayscale, (3) DirectGrayscale, (4) Frequency Bursts, (5) Multi-Pulse. The Test Patterncomponents also include special Geometry Reference Markers, and somemore optional enhancement components on a flat color background, e.g.50% Gray, such as Vertical Resolution Wedges and/or Radial Mires and/orTiming Reference dynamic components, e.g. clock dial or current videoframe number display. The Geometry Reference Markers within the TestPattern are implemented as several (typically four) small circles,filled with two contrast colors, e.g. White and Blue, thus providing forreliable differentiation of the said Markers from the rest of TestPattern and accurate positioning of said circles centers locationswithin the captured video frame. The Geometry Reference Markers arelocated at four corners of the rectangle derived by the downscaling ofthe Test Pattern outer boundary with some known fixed scaling factor,e.g. 0.75. The XY co-ordinates of all Test Pattern components within isvideo frame are re-mapped for measurement purposes from their original(ideal) positions to their actual positions using the scaling and offsetcoefficients based on previously measured XY positions of ReferenceMarkers. One or more Color Bars Band can be shown in two versionsdiffering in color saturation: (1) full saturation version for TestPattern File, i.e. for signal/data processors testing, and (2) reducedsaturation version for Reflectance Chart, i.e. for video camerastesting. The Grayscale Bands can include optional Black Shallow RampInsert (“Near-Blacks”) and/or White Shallow Ramp Insert (“Near-Whites”),purposed for more accurate YRGB Range Black Level Overload and YRGBRange White Level Overload measurements. The video quality analysisstarts with the detection of the Reference Markers relative positionswithin the captured video frame (also used as a prove of Test Patternand Test Setup validity) and finishes with the creation of the ReportFile(s) including the results of all measurement steps and Summary Tableshowing the Results Values in line with the user-defined Target Values.The number of video quality analysis steps in the said multi-stepprocess depends on the detection of the valid Reference Markers relativepositions, i.e. on Test Pattern and Test Setup validity. In case ofsuccessful detection of valid Reference Markers the video qualityanalysis include Image Geometry Measurements, Pulse ResponseMeasurements, YUV/YRGB Levels measurements, Y Gamma and YRGB RangeOverload Measurements, Frequency Response Measurements and NoiseMeasurements; but in case of unsuccessful Reference Markers detectionthe analysis process collapses to Noise Measurements only. A YUV/YRGBLevels analysis include the comparison of actual measured levels with aset of pre-calculated Reference Levels, whilst these Reference Levels inturn depend on automatic Test Chart Type detection (differing in fullsaturation vs. reduced saturation) and manual or automatic Color Schemeselection—two most important Color Schemes are “0-255” scheme, usedmainly in computer graphics applications, and “16-235” scheme, commonlyused in video applications. An automatic Color Scheme selection is basedon the comparison of actual measured RGB levels with several sets ofReference Levels, each set representing Color Bars values for one ColorScheme; result is the selection of the Scheme providing for the smallestmaximal error (minimum distance in the RGB color space). An automaticTest Chart Type selection can based on the comparison of actual measuredRGB Color Bars levels with two sets of pre-calculated Reference Levelsand the results of Color Saturation measurement based on the comparisonof relative gain of Colored Pulse components in the Multi-Pulse Band—Ygain vs. UV gain.

Advantages of the preferred embodiments may include one or more of thefollowing. The system uses EXACTLY THE SAME test pattern for cameras andvideo processors alike. This is more convenient than other systems thatuse EITHER video signal test matrix, suitable ONLY for video dataprocessors, OR reflectance charts, suitable ONLY for cameras. These twotypes of test patterns traditionally used by other systems for two typesof applications have no similarity at all. The system uses only ONE testpattern for a variety of opto-electronic systems, such as teleconferencesystem. This universality allows users to insert and capture test dataat any point in the signal processing chain—from camera lens to the verylast decoder.

Other advantages of the preferred embodiments may include one or more ofthe following. The system accurately characterizes the most importantobjective parameters of video processing quality such as:

-   -   picture geometry described in simplified form as picture        position and size    -   video levels traditionally expressed in picture brightness,        contrast, saturation and RGB values    -   video image uniformity usually described in terms of horizontal        and vertical “shading”    -   picture sharpness traditionally represented by pulse and        frequency response values    -   analog and digital noise artifacts traditionally represented by        SNR values

Objective measurement of the parameters listed above allows practicalobjective judgment on picture quality or more precisely “loss of qualityin video processing workflow”. For example, the proposed system can beused for a variety of applications to find in advance video imagedistortions associated with particular profile of video camera, videoformat conversion device and/or video compression codec. The systemallows drastic improvement of speed, sophistication and completeness ofautomated video quality analysis. The system can createresolution-agnostic video quality metrics and a testing methodology forobjective measurements of offline or online video processing pathwithout any referral to particular live video content, but covering allpractically used steps of this content processing—from the camera lensto the destination side video display input. The system is applicablefor modern multi-format teleconference and content delivery environment.

Any consumer or professional system or device that has the ability toprocess video images or video data in order to deliver and/or displayvideo and/or other multi-media content can use the objective measurementsystem. The system can also be beneficial for benchmarking purposes,e.g. for comparison of different cameras or compression codecs orcomparison of different encoding profiles of the particular encoder. Thesystem is especially useful where the data processing services areutilizing file-based environment for the preparation and delivery ofvideo content. The system provides for fast and accurate analysis of alllisted parameters. The software reliably works within the wide range ofvideo image conditions in terms of image size and position, relativelybig geometry errors, lighting non-uniformities and in presence ofrelatively high embedded noise.

BRIEF DESCRIPTION OF THE DRAWINGS

This system will now be described by way of example with reference tothe accompanying drawings in which:

FIG. 1 shows variants of analysis workflow.

FIG. 2 illustrates an exemplary test pattern composition.

FIG. 3 shows an exemplary Software Workflow Diagram for a Video qualityanalyzer.

FIG. 4 shows an exemplary Test Result Summary Table for 1920×1080 imageresolution.

FIG. 5 shows an exemplary Test Result Summary Table for 1280×720 imageresolution.

FIG. 6 shows an exemplary Geometry Test Result for Reflectance Chart.

FIG. 7 shows exemplary Video Level Test Results.

FIG. 8 shows exemplary details of Near-White Test Pattern used at LevelsTest stage.

FIG. 9 shows exemplary Frequency Response Test Results.

FIG. 10 shows exemplary Noise Analysis Test Results.

DESCRIPTION

The following description of the present system is done by the way ofnon-exclusive example of how the Video quality analyzer would work in anenvironment where video content is distributed through a video datadelivery service such as a videoconferencing system.

One embodiment of the analyzer is software that runs on hardware orcomponents readily available on the market. In the preferred embodiment,the present system consists of a standard off-the-shelf video capturedevice, e.g. Unigraf capture card, and software executable running understandard OS, e.g. Microsoft Windows. The system can determine the videoquality of digital SD or HD TV and IPTV data processing cases, inparticular—video cameras, compression codecs, scalers, TV sets, STBs,PCs, or portable devices.

The system performs automated Measurement of Video Quality parameters bya static Test Pattern provided either in form of a Test Pattern File,converted via a standard playout device (test source) into analog ordigital test signal and supplied to the input of System Under Test, orin form of a Reflectance Chart installed before the front-end device ofthe System Under Test, such as TV camera. The test pattern is recordedas a data file by a video capture device connected to the back-enddevice of the System Under Test; e.g. to the output of systemdecoder/player. The test pattern is then analyzed by a video qualityanalyzer that in turn generates a detailed video quality AnalysisReport.

Referring initially to FIG. 1, a reflectance chart 90 is captured by acamera 92 whose output is directed connected to a computer (USB orFireWire) or through a capture card 110. Alternatively, a test file 100can be played by a reference player 102 and provided to a videoprocessor 106. The test file 100 can also be played by a video player104. The output of video player 104 or video processor 106 can becaptured by the capture card 110. The test file 100 can also be encodedby a video encoder 108, and decoded by a reference decoder 112 if thequality of the encoder/decoder is being tested. The output of thecapture card 110 or the decoder 112 is a video file 114 that can beanalyzed by a video quality analyzer 120 which generates report 130 thatcontains diagnostics data and data describing the quality of the video.The result is an Objective Picture Quality Metering System withpractical application to software, hardware or hybrid devices. Thesystem's measurement results contribute to the improvement of perceivedquality of static or dynamic digital pictures.

One purpose of the video quality analyzer is to measure captured videofiles from any HD or SD source. With the Reflectance Chart 90, thesystem measures video cameras, but through a video player, processorsand codecs the system can measure the overall performance of complex andsophisticated video data transmission chains.

FIG. 2 illustrates an exemplary composition of a Test Pattern matrix. Inthis embodiment, a static matrix test pattern provides for automatedmeasurement algorithms of all relevant video parameters. This testpattern could be also combined with live video content to provideobjective video quality reference points along full video distributionchain—from content origination, through content re-purposing anddistribution to final content consumption at the consumer display.Preferably, the same test pattern, both in form of optical reflectancechart and in form of video signal source, is used for consistent testingof full chain from camera lens to the display screen.

Turning to FIG. 2, each of five Bands from #1 to #5 is dedicated to aparticular sub-set of video quality parameters. Band #0 contains severaloptional visual components, which are not related to automatic analysis.The test patterns also include Geometry Reference Markers 10, whichprovide for features such as:

1. Geometry checks, such as test chart scaling (zoom), XY offset(position); in case of Reflectance Chart they also serve to measure tiltand keystone parameters

2. Test Pattern Validation: if Reference Markers are not present (notdetected) analysis process collapses to Noise Measurement only

3. All other measurements are using scaling/positioning coefficientscalculated from the detected Reference Markers positions within thevideo frame.

The test pattern also includes a component 20 that provides forsharpness/spatial shading determination.

FIG. 3 shows an exemplary process to determine video quality. The firststep 302 is Test Case configuration. At this stage User selects assumedYUV file format and optionally the assumed YRGB Range selection. Thisdoes not require any significant changes in the data processingalgorithm, but may drastically change the presentation andinterpretation of the analysis results.

Next, step 304 consists of data source selection: either live data fromthe capture card 306 via the driver 308, or pre-captured video datastored in the file 310. In both cases video data can be presented eitherin YUV (UYVY) format or in RGB format, among others. In one embodiment,the selection is stored in form of YUV/RGB Flag, used in all furthercalculations.

The result of the acquisition step 304 is large array of video data 312,which can be single video frame or small group of video frames, e.g.eight consecutive frames; this array should be processed during thesubsequent steps. The size of this array must be large enough toaccommodate the data. For example, at 1920×1080 resolution the requiredYUV data array size in bytes is 1920×1080×2×8=approximately 33 MB of RAMin one embodiment.

An optional Viewer and Waveform Scope module 314 allows user to previewincoming images and YUV/RGB waveforms of any line or averaged group ofinput video lines. The scope feature is useful in finding out thereasons of automated analysis failure, e.g. it may be caused by theincoming video data timing errors.

At step 316, in one embodiment, acquired data from all available videoframes are first averaged to reduce harmful effect of embedded noise.The test pattern image is then split into four quadrants; each quadrantis searched at step 316 for the presence and position of ReferenceMarkers 10 (FIG. 2).

At step 318, geometry test parameters, such as H & V position offset, H& V scaling coefficients, effective chart size (which can be smaller orbigger than video frame size), image tilt, keystone distortion, are puttogether and presented in a Geometry Test Report using predeterminedcommonly accepted units, e.g. in pixels and/or percents of image height.These parameters can be mathematically calculated based on differencesbetween ideal and measured positions of Reference Markers, among others.

If all four Reference Markers are found in approximately correctpositions, then a Test Pattern Validation Flag is activated. This flagis used in the Geometry Test Report and also serves to enable severalfurther stages of automated analysis.

Acceptable marker positions cover wide range of scalingcoefficients—from 110% down to 45% in one embodiment. However, the rangeof permitted offsets, tilts and keystone values should be rather small.For example, if chart image tilt exceeds 10 degrees, the rectangulararrangement of color patches within the Test Pattern Bands issignificantly deteriorated. In such case the Validation Flag should bedeactivated. Significant H or V offset also may cause complete loss ofsome test pattern components, so large offset values should be avoided.Thus, linear scaling (zoom) is permitted within reasonable limits, butother geometry transformations should be restricted.

All further steps rely on steps 316 and 318 in terms of re-mappedpositions of all measurable components within the Bands #1, 2, 3, 4 and5. For example, to find the co-ordinates of Frequency Burst labeled “1”and located on the Band #4 left side, its original (default) positionshould be re-mapped proportionally to the measured offsets of ReferenceMarkers—as illustrated on FIG. 6.

If valid Test Matrix Pattern is not detected, then the whole analysisprocess collapses to Noise Test only—following the workflow control step320. As described below, noise measurement does not rely on ReferenceMarkers, so noise can be measured on any static image, such as fullscreen color bars or just flat full field color, e.g. gray field, amongothers. All results of step 318 are summarized in the Geometry TestReport.

At step 322 Band #5 (Pulses and Bars) is analyzed, resulting in K-factorvalue, measured on “white needle” pulse, and Y vs. UV Gain (Saturation),measured on soft green pulse.

The Y_vs_UV_Gain value is important. First, it describes general imagequality deterioration—color saturation loss or excessive boost. Second,together with the color bars levels measurement results, it provides forautomatic switching between two modes of operation of the video qualityanalyzer: “optical” reflectance chart mode and “electric” test patternmode. This switch is created and applied later—at step 324. All resultsof step 322 are summarized in Pulse Response Test Report.

Step 324 consists of preliminary setting of modes of operation andfinding the important general parameters, such as luminance signaldynamic range on Band #2 (Inverted Grayscale) and Band #3 (Grayscale),prior to detailed levels analysis applied at next step 326.

For greater robustness, Band #2 is mirrored and luminance values of twobands are averaged, thus minimizing harmful effects of non-uniformlighting—e.g. if lighting level linearly drops from left to right, thenhalf-sum of left and right white patches levels is exactly equal to thewhite level in the middle portion of the picture.

One of the step 324 goals is distinguishing between two possible ColorSchemes: 0-255 scheme, used mainly in computer graphics applications,and 16-235 scheme, commonly used in video applications.

The manual or automatic selection of Color Scheme is important in oneembodiment because it affects the assumed nominal values of all colorsin all bands. The selection of wrong Color Scheme may jeopardize allcolor analysis results. The comparison of actual color bars saturation,measured at step 324, with the Y_vs_UV_Gain value, measured at theprevious step 322, allows distinguishing between “optical” reflectancechart mode and “electric” test pattern mode.

In optical mode color bars saturation is about 6 dB lower thanY_vs_UV_Gain value; in “electric” mode they should be approximatelyequal. This mode switching is needed to select appropriate referenceYUV/RGB color bars values used to calculate color errors table.

At step 324 the Y channel “candidate” levels on black and white patchesare tested against the decision thresholds set half-way between possiblenominal values. The Default Scheme is 16-235 (“Video”). If average Yvalue on black patch is below 0.5*(0+16) and measured Y value on whitepatch is above 0.5*(255+235), then the Color Scheme 0-255 (“ComputerGraphic”) is selected.

At step 326 Bands #1 (Color Bars), #2 (Inverted Grayscale) and #3(Grayscale) are split into several rectangular areas (patches); eachpatch contains only one color; examples are Yellow patch within theColor Bars or 100% White patch of Inverted Grayscale.

Video data within the central portion of each patch are averaged forfurther suppression of noise and other artefacts. This results inmeasured YUV and RGB values for all patches.

At step 326 the YUV/RGB Flag, set at step 304, is used to control thedirection of color space conversion—either derivation of RGB values fromYUV values (if acquired data are in YUV format) or vice versa derivationof YUV values from RGB values (if acquired data are in RGB format). YUVand RGB values of all patches are further processed to calculatestandard colorimetric parameters, such as Black and White levels,Luminance Gamma, Dynamic Color Balance Errors, YUV and RGB values ofColor Bars, etc. This includes application of well-known standard colorspace conversion coefficients and formulae. For faster finding oferroneous color values in the presented result tables, special highlightflags can be created, marking the colors with maximal absolute errors(maximal RGB space distance from the correct nominal values).

Near-White and Near-Black inserts within the Bands #2 and #3 requirespecial processing. Unlike other color patches, these components of testpattern contain linear gradients, so called shallow ramps.

YRGB range overloading, e.g. caused by excessive opening of camera'siris or by “black level crash” of video processor, can be detected inform in clipping of these shallow ramps.

The size of clipped area is directly proportional to the overloadstrength (percentage of lost dynamic range). Count of clipped pixels,divided by the total number of pixels in this test pattern component,represents the percentage of detected overload. All results of step 326are summarized in Levels Test Report.

At step 328 central portions of all frequency bursts within Band #4 aremeasured. This creates six pairs of arrays containing peak and troughvalues. Differences between peaks and troughs are averaged, thus findingout the average contrast of each burst. The contrast values are thenreferenced to interpolated contrast of dark gray and light gray patcheson both sides of the Band.

Relative contrast values represent individual bursts positions along thevertical axis of Frequency Response Plot shown on FIG. 9.

Horizontal axis positions on the Frequency Response Plot, i.e. actualfrequencies of the captured bursts, are calculated by scaling original“pristine” burst frequencies in accordance with the H scalingcoefficient measured at step 318. All results of step 328 are summarizedin Frequency Response Test Report.

Step 330 (noise analysis) includes several stages of spatial andtemporal filtering. The goal of this filtering is the separation ofrandom noise YUV values from static YUV values of the test patternitself. Important feature of this filtering is the preservation of noisehorizontal spectrum shape.

Because any horizontal filtering is undesirable, noise separationprocess consists of vertical-temporal high-pass filtering. The firststage is temporal filtering, achieved by deduction of the central frameYUV values from the average YUV values across eight adjacent videoframes. The second stage is vertical filtering. Many modern videoprocessors involve video line averaging; typical vertical aperture sizeof such processor is from two to five video lines. This may produce anoise of specific type—highly correlated in vertical dimension. Accuratemeasurement of such noise requires vertical filters with the aperturesize much larger than the incoming noise vertical correlation interval.

This is implemented at step 330 by adding together the powers (energy)of YUV differences taken across eight video lines. Eight TV lines is bigenough vertical distance, allowing to overcome the abovementionedproblem of vertically correlated video noise handling.

Filtered out noise, separate values for Y, U, V, R, G, and B channels,is then processed by standard statistical formulae, resulting instandard deviations, histograms and Y noise horizontal spectrumplots—with and without weighting filters.

The separated noise values are presented as a viewable image with“boosted” contrast. This noise image together with the horizontalspectrum plot allows advanced user to distinguish truly random noisefrom periodic interferences, such as cross-talks or digital clockpick-ups. The results of step 330 are summarized in Noise Test Report.

At every step for user convenience all partial test result reports arepresented as plots and tables on several separate pages (windows), i.e.Geometry Page, Levels Page, among others.

At step 332 the most important Test Results are compared with theuser-defined target values and presented in three formats:

1. On-screen Results Summary Table 334

2. Detailed printable Report 336, e.g. PDF file

3. Machine-readable Report 338, e.g. Excel spreadsheet file.

FIGS. 4-10 show exemplary test results.

FIG. 4 shows an example of a Test Results Summary Table for 1920×1080image resolution. The Summary Table shows the measured video qualityparameters and corresponding target values (user-defined tolerances). Ifthe measured result is within the target range, then this row of thetable shows green tick (pass mark) in the Pass/Fail column. If themeasured result is outside of the target range, then green tick isreplaced by a red cross (failure mark). The scoring of these pass/failmarks provides for fully automated (unattended) analysis mode. Forexample, in the strictest variant, appearance of just one red cross inany row means that system or device does not pass the test. A thumbnailpicture at the bottom of the Summary Page serves mainly for quick visualestimate of general test conditions. For example, significantReflectance Chart tilt or lighting non-uniformity may invalidate alltest results.

FIG. 5 shows an example of Test Results Summary Table for 1280×720 imageresolution. The main difference against FIG. 4 is the size of thethumbnail picture; this display allows quick visual check of actualvideo data resolution.

FIG. 6 shows one example of Geometry Test Results for Reflectance Chart.In this example, the inner corners of four green squares indicatecalculated positions of the four corners of Test Chart. Their positionsare calculated by extrapolation of the measured positions of fourblue-white Reference Markers. Despite the fact that upper-left andbottom-left corners are not visible their calculated positions arecontributing to the final results.

Turning now to FIG. 7, an example of Video Levels Test Results is shown.The page contains many partial parameters; together they givecomprehensive presentation of Y, R, G, and B gradations rendition andinter-channel misbalances. Black Level and White Level are presented in% of the selected nominal YRGB range and also in D18 bit levels.Luminance Gamma is calculated by best fitting method on 9 of 11staircase porches; two lowest porches are ignored to minimize noise andglare related effects. RGB Dynamic Balance Error is a maximum of R-G,B-G and R-B magnitudes of all 11 staircase porches. Black Balance Errorand White Balance Error are calculated similarly, but only the lowest(Black) and the highest (White) porches are used. Black Crash and WhiteCrash (Y Range Overload) are measured by finding the clipping level ofshallow ramps in the central area of the Test Pattern. The bottom halfof the page is occupied by Color Bars Table. It contains YUV and RGBlevels of the test pattern measured on Band #1. The Table also shows (inGray) the reference values of 100/0/75/0 Color Bars corresponding to theselected Nominal Range (16-235 or 0-255). The right half of each cellshows calculated Color Bar Errors, i.e. differences between measured andreference values. Video data can come as YUV or as RGB. Values withinthe “Captured Data” part of the Table are YUV or RGB data, averaged androunded to 8 bit values without any mapping or scaling. Values withinthe “Derived Values” part of the Table are results of application ofstandard Color Space Conversion Matrix to the input data; these resultsare also rounded to 8 bit and compared with the corresponding 8 bitreference values.

Referring now to FIG. 8, exemplary details of Near-White Test Patternused at Levels Test stage are shown. In this example all three shallowramps of R, G and B channels are not distorted—the ramp waveforms arelinear and not clipped.

FIG. 9 shows another example of Frequency Response Test Results. Thispage shows the measured averaged peak-to-peak amplitudes of sixfrequency bursts and display of averaged luminance waveform ofmulti-burst band (Band 4). The burst amplitudes are expressed in dB withrespect to nominal (undistorted) value. This band of test patternincludes special reference bars with levels exactly matching the nominalburst amplitude. The measurement algorithm checks these bars first, andautomatically compensates for any non-standard Black Level and WhiteLevel conditions, including Levels Tilt. This allows the frequencyresponse measurement to be always accurate and correct, independent ofany lighting, setup or gain errors in Y channel. Frequencies are shownabove the response values in two formats: original (pristine chart)values are shown in gray, and actual scaled values in black. In theexample shown they differ only slightly because the camera zoom (98%) isvery close to 100%. Scaled frequency values are also used for plottingthe response curve at the bottom of the page. On the frequency responseplot the measured values are shown in blue, target limits are shown inbrown.

FIG. 10 shows yet another example of Noise Analysis Test Results. Themost important noise parameter is RMS noise level of Y channel displayedin the upper left corner of the page using three types of units:

-   -   a) % of Nominal White    -   b) D1 8 bit levels (in brackets)    -   c) Equivalent mV of analog Y signal (also in brackets)

Other important noise parameters present on this display are:

-   -   a) Y channel SNR, calculated in three variants: unfiltered,        band-limited and weighted. The Y RMS value (shown above the Y        SNR) directly correlates with unfiltered Y SNR.    -   b) UV SNR, derived from band-limited unweighted sum of scaled U        noise and V noise    -   c) R, G, B and “Dark B” SNR values, derived from Y and UV SNRs

A histogram display in the upper right corner allows differentiationbetween truly random Gaussian (i.e. unprocessed) noise and “cored” noisesignal typically produced by noise reducers. If Y and G histogram plotsare close to ideal Gaussian curve shown in gray, then the effect ofnoise reduction is rather small.

In the example shown the difference between two curves is very large,which indicates that camera processor applied very deep noise reductionreducing the relative probabilities of low noise magnitudes vs. highmagnitudes.

Bottom part of the page contains Y Noise Spectral Density plots indB/MHz for unlimited and weighted noise spectra and also Noise Imagewith boosted contrast. These two displays allow to see the effect ofdevice under test frequency response and also to distinguish randomnoise from the contributions by regular textures, e.g. from those causedby RF interference or digital clock pick-up.

Various modifications and alterations of the invention will becomeapparent to those skilled in the art without departing from the spiritand scope of the invention, which is defined by the accompanying claims.It should be noted that steps recited in any method claims below do notnecessarily need to be performed in the order that they are recited.Those of ordinary skill in the art will recognize variations inperforming the steps from the order in which they are recited. Inaddition, the lack of mention or discussion of a feature, step, orcomponent provides the basis for claims where the absent feature orcomponent is excluded by way of a proviso or similar claim language.

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not of limitation. Likewise, the various diagrams maydepict an example architectural or other configuration for theinvention, which is done to aid in understanding the features andfunctionality that may be included in the invention. The invention isnot restricted to the illustrated example architectures orconfigurations, but the desired features may be implemented using avariety of alternative architectures and configurations. Indeed, it willbe apparent to one of skill in the art how alternative functional,logical or physical partitioning and configurations may be implementedto implement the desired features of the present invention. Also, amultitude of different constituent module names other than thosedepicted herein may be applied to the various partitions. Additionally,with regard to flow diagrams, operational descriptions and methodclaims, the order in which the steps are presented herein shall notmandate that various embodiments be implemented to perform the recitedfunctionality in the same order unless the context dictates otherwise.

Although the invention is described above in terms of various exemplaryembodiments and implementations, it should be understood that thevarious features, aspects and functionality described in one or more ofthe individual embodiments are not limited in their applicability to theparticular embodiment with which they are described, but instead may beapplied, alone or in various combinations, to one or more of the otherembodiments of the invention, whether or not such embodiments aredescribed and whether or not such features are presented as being a partof a described embodiment. Thus the breadth and scope of the presentinvention should not be limited by any of the above-described exemplaryembodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; the terms “a” or“an” should be read as meaning “at least one,” “one or more” or thelike; and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

A group of items linked with the conjunction “and” should not be read asrequiring that each and every one of those items be present in thegrouping, but rather should be read as “and/or” unless expressly statedotherwise. Similarly, a group of items linked with the conjunction “or”should not be read as requiring mutual exclusivity among that group, butrather should also be read as “and/or” unless expressly statedotherwise. Furthermore, although items, elements or components of theinvention may be described or claimed in the singular, the plural iscontemplated to be within the scope thereof unless limitation to thesingular is explicitly stated.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “module” does not imply that the components or functionalitydescribed or claimed as part of the module are all configured in acommon package. Indeed, any or all of the various components of amodule, whether control logic or other components, may be combined in asingle package or separately maintained and may further be distributedacross multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives may be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

The previous description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these embodiments will be readilyapparent to those skilled in the art, and the generic principles definedherein may be applied to other embodiments without departing from thespirit or scope of the invention. Thus, the present invention is notintended to be limited to the embodiments shown herein but is to beaccorded the widest scope consistent with the principles and novelfeatures disclosed herein.

What is claimed is:
 1. A video monitoring system to perform automatedMeasurement of Video Quality parameters, comprising: a static testpattern provided as a test pattern file rendered by a device under testor a reflectance chart captured by a camera under test, wherein the testpattern file is identical to the reflectance chart and includes amultitude of video test components combined in one test patternincluding horizontal test bands forming a multi-row matrix, each testband containing test pattern components specific for a particularsub-set of video quality parameters; a video capture device coupled tothe device under test or camera under test to generate video data foranalysis; and a video quality analyzer processor processing the videodata into an analysis report, wherein color co-ordinates of capturedimage test components are compared with known reference values.
 2. Thesystem of claim 1, wherein the test pattern contains video componentsfor (1) aural and visual estimation, (2) on-line or off-lineinstrumental analysis, and (3) fully automated on-line or off-lineanalysis.
 3. The system of claim 1, wherein the test pattern includes amultitude of video test components combined in one test patternincluding horizontal test bands forming a multi-row matrix, each testband containing test pattern components specific for a particularsub-set of video quality parameters.
 4. The system of claim 1, whereinthe test pattern includes test bands consisting of Color Bars, InvertedGrayscale, Direct Grayscale, Frequency Bursts, and Multi-Pulse.
 5. Thesystem of claim 1, wherein the test pattern components include severalcircular Geometry Reference Markers, and optional enhancement componentson a flat color background.
 6. The system of claim 5, wherein theGeometry Reference Markers within the Test Pattern comprise circles,filled with two contrast colors providing for reliable differentiationof said Markers from the rest of test pattern and accurate positioningof circle centers locations within a captured video frame.
 7. The systemof claim 5, wherein the Geometry Reference Markers are located at fourcorners of a rectangle derived by downscaling of a Test Pattern outerboundary with a fixed scaling factor.
 8. The system of claim 1, whereinXY co-ordinates of test pattern components within a video frame arere-mapped for measurement purposes from original positions to actualpositions using scaling and offset coefficients based on previouslymeasured XY positions of Reference Markers.
 9. The system of claim 1,comprising one or more Color Bars Band shown in two versions differingin color saturation: (1) full saturation version for Test Pattern Fileand (2) reduced saturation version for Reflectance Chart.
 10. The systemof claim 1, comprising one or more Grayscale Bands with one of: a BlackShallow Ramp Insert (“Near-Blacks”), and a White Shallow Ramp Insert(“Near-Whites”), purposed for YRGB Range Black Level Overload and YRGBRange White Level Overload measurements.
 11. The system of claim 1,wherein the video quality analyzer starts with a detection of ReferenceMarkers relative positions within a captured video frame and finisheswith a creation of Report File(s) including results of measurement stepsand Summary Table showing Results Values in line with user-definedTarget Values.
 12. The system of claim 1, wherein the number of videoquality analysis steps depends on a detection of valid Reference Markersrelative positions.
 13. The system of claim 1, wherein after successfuldetection of valid Reference Markers, the video quality analysisincludes Image Geometry Measurements, Pulse Response Measurements,YUV/YRGB Levels measurements, Y Gamma and YRGB Range OverloadMeasurements, Frequency Response Measurements and Noise Measurements.14. The system of claim 1, wherein after unsuccessful Reference Markersdetection, the video quality analysis comprises Noise Measurements only.15. The system of claim 1, wherein the video quality analyzer performs aYUV/YRGB Levels analysis including a comparison of actual measuredlevels with a set of pre-calculated Reference Levels, whilst theseReference Levels in turn depend on automatic Test Chart Type detection(differing in full saturation vs. reduced saturation) and manual orautomatic Color Scheme selection.
 16. The system of claim 1, wherein anautomatic Color Scheme is selected based on a comparison of actualmeasured RGB levels with several sets of Reference Levels, each setrepresenting Color Bars values for one Color Scheme to select a Schemeproviding for the smallest maximal error (minimum distance in the RGBcolor space).
 17. The system of claim 1, wherein the video qualityanalyzer performs automatic Test Chart Type selection based on acomparison of actual measured RGB Color Bars levels with two sets ofpre-calculated Reference Levels and the results of Color Saturationmeasurement based on a comparison of relative gain of Colored Pulsecomponents in the Multi-Pulse Band—Y gain vs. UV gain.
 18. The system ofclaim 1, wherein the test pattern contains video components for (1)aural and visual estimation, (2) on-line or off-line instrumentalanalysis, and (3) fully automated on-line or off-line analysis, whereinthe test pattern includes a multitude of video test components combinedin one test pattern including horizontal test bands forming a multi-rowmatrix, each test band containing test pattern components specific for aparticular sub-set of video quality parameters, wherein the test patternincludes test bands consisting of Color Bars, Inverted Grayscale, DirectGrayscale, Frequency Bursts, and Multi-Pulse, wherein the test patterncomponents include Geometry Reference Markers, and optional enhancementcomponents on a flat color background, and wherein the GeometryReference Markers within the Test Pattern comprise circles, filled withtwo contrast colors providing for reliable differentiation of saidMarkers from the rest of test pattern and accurate positioning of circlecenters locations within a captured video frame.
 19. The system of claim1, wherein after successful detection of valid Reference Markers, thevideo quality analyzer performs Image Geometry Measurements, PulseResponse Measurements, YUV/YRGB Levels measurements, Y Gamma and YRGBRange Overload Measurements, Frequency Response Measurements and NoiseMeasurements and wherein, after unsuccessful Reference Markersdetection, the video quality analyzer performs Noise Measurements only.20. The system of claim 1, wherein the video quality analyzer performs aYUV/YRGB Levels analysis including a comparison of actual measuredlevels with a set of pre-calculated Reference Levels, whilst theseReference Levels in turn depend on automatic Test Chart Type detection(differing in full saturation vs. reduced saturation) and manual orautomatic Color Scheme selection and wherein an automatic Color Schemeis selected based on a comparison of actual measured RGB levels withseveral sets of Reference Levels, each set representing Color Barsvalues for one Color Scheme to select a Scheme providing for thesmallest maximal error (minimum distance in the RGB color space).